STDFormer: Spatial-Temporal Motion Transformer for Multiple Object Tracking

Mainstream multi-object tracking methods exploit appearance information and/or motion to achieve interframe association. However, dealing with similar and occlusion is a challenge for information, while limited by linear assumptions prone failure in nonlinear patterns. In this work, we disregard clues propose pure tracker address the above issues. It dexterously utilizes Transformer estimate complex achieves high-performance low computing resources. Furthermore, contrastive learning introduced optimize feature representation robust Specifically, first long-range modeling capability of mine intention temporal decision spatial interaction introduce prior detection constrain range estimation. Then, as an auxiliary task extract reliable features compute affinity bidirectional matching improve computation distribution. addition, given that both tasks are dedicated narrowing embedding distance between tracked object features, design joint-motion-and-association framework unify two one optimization. The experimental results achieved three benchmark datasets, MOT17, MOT20 DanceTrack, verify effectiveness our proposed method. Compared state-of-the-art methods, STDFormer sets new on DanceTrack competitive performance MOT17 MOT20. This demonstrates advantage method handling associations under appearance, or motion. At same time, significant advantages over Transformer-based learning-based suggest direction application MOT. generalization unmanned aerial vehicle (UAV) videos, also evaluate VisDrone2019. show VisDrone2019, which proves it can handle small-scale UAV videos well. code available at https://github.com/Xiaotong-Zhu/STDFormer.